Skip to content

feat(plugins): add a Cloudflare R2 SQL driver - #2030

Merged
datlechin merged 12 commits into
mainfrom
feat/cloudflare-r2-sql
Sep 11, 2026
Merged

feat(plugins): add a Cloudflare R2 SQL driver#2030
datlechin merged 12 commits into
mainfrom
feat/cloudflare-r2-sql

Conversation

@datlechin

@datlechin datlechin commented Aug 6, 2026

Copy link
Copy Markdown
Member

Adds a registry-only driver for Cloudflare R2 SQL, Cloudflare's read-only SQL engine over the Apache Iceberg tables in an R2 bucket. Connect with an Account ID, a bucket and an API token. The sidebar lists Iceberg namespaces as schemas with their tables under them. Queries run in the editor, tables browse and filter in the grid, and export works. Nothing in the app offers a write.

The first version of this PR did not work against the live API, and its two app-side changes had the wrong shape. This revision rewrites the driver and replaces both app changes. It lands as three commits, a fourth with the review fixes, a merge of main, and two follow-ups (sections 4 and 5). Each commit builds and passes its tests on its own.

1. fix(connections): read-only engines and remote files are held at Read-Only

Before: read-only was pushed into the user's Safe Mode setting from one place, the form's type-change hook. A connection reaches Safe Mode four other ways: session creation, the toolbar and Database > Safe Mode Level menu (Silent was still selectable on R2), the reconcile after a connection update, and the MCP policy snapshot. Switching the form's type away from R2 also left Read-Only saved on the new connection. The existing precedent, remote SQLite files over SSH (#2553), had the same hole: the forced level lived only on the rewritten connection, which nothing reads, so the grid, the editor, the AI tools and MCP all accepted edits to a local copy that is never written back.

Now: DatabaseConnection.safeModeLevel is the level in force. That is the user's own choice (preferredSafeModeLevel, the stored field), raised to the connection's SafeModeFloor. Read-Only is the floor for a read-only engine (Cloudflare R2 SQL, Beancount) or a remote database file. Section 5 adds the profile's minimum. Every existing reader gets the enforced level without changes of its own, and if a save path were ever missed it would store a stricter level, not a weaker one. Storage, sync, connection export and the form read and write preferredSafeModeLevel, so an engine fact never overwrites the user's own setting. The execution gate is back to its main shape. The form shows Safe Mode as fixed text with the reason in the section footer, not a disabled picker bound to the stored value, and the Safe Mode menus dim every other level. Picking the one level left changes nothing, so it does not overwrite the saved level either.

2. refactor(datagrid): engines that cannot skip rows

Before: a supportsOffsetPagination Bool with guards in scattered places, plus a new "First %d rows" string that brought back the "1 rows" defect and repeated the readout next to it. It missed menu validation, the page-size presets, All rows…, Custom…, MCP browse_table (which still sent OFFSET) and export (silently capped at 10,000, or R2's 500-row default without the plugin's query). It also missed query tabs: a query with no LIMIT came back at R2's default of 500 rows, labelled as the whole result.

Now: a PaginationCapability (.offset or .leadingRowsOnly(maximumRows:)), curated on the metadata registry. PaginationStyle is @frozen, so this adds no PluginKit change and needs no ABI bump.

  • TableQueryBuilder takes the capability as a required parameter, so every construction site states it. It emits a clamped LIMIT and never OFFSET.
  • PaginationCoordinator sends every page move through one gate and clamps page size. The page size is also clamped where a tab is created and where one is restored.
  • ResultStatusModel decides showsPageNavigation. The existing readout already says Rows 1-500 or 1-500 of N rows, so no new string.
  • The rows-per-page presets and Custom… stop at the maximum, All rows… is hidden, and Query > First/Previous/Next/Last Page are dimmed.
  • No automatic COUNT(*) on an engine that bills per scan. Count Exactly is still there on demand.
  • LeadingRowsStatement gives every read the user did not limit an explicit LIMIT: the row cap plus one row, or the maximum for Fetch All, exports and external clients. The engine's smaller default never applies.
  • MCP browse_table refuses offset > 0 and clamps limit. Export, query-result export and table transfer each warn when they are cut short at the maximum. A result that reaches it says so in the status bar and in MCP's status_message.

3. fix(plugin-cloudflare-r2-sql): the driver

  • Wire contract. Live responses carry each column's type under schema[].descriptor.type.name, per a Cloudflare engineer's wrangler test fixture (dqe-prod-test), Cloudflare's own skills reference, and a shipping third-party client. The old decoder read schema[].type, so every type came back empty and binary was never decoded. Decoding is strict now: a body that does not match is a readable error, not an empty result.
  • Exact numbers. Values decode as Decimal. Measured on this toolchain, the old Int64-first decode turned 12345678901234567.89 into 12345678901234567 with no error, and turned 1e400 into NULL.
  • Metadata read by column name. DESCRIBE uses its documented columns: nullable is !required, and the comment is doc. The old code read required as nullability, so every optional column showed NOT NULL. SHOW output is undocumented and Spark and DataFusion disagree on column order, so it reads the known name columns and reports any other shape rather than guessing.
  • Transport. URLSession.data(for:delegate:) with a per-request task set, now in TableProR2SQLCore so it is testable. Measured: the old one-slot transport cancelled the latest request (often a sidebar read) instead of the query, and ignored Task cancellation. The user's Query Timeout and export's timeout suppression now apply; before, the timeout was hard-coded at 60 s.
  • Removed. The plugin's own browse/filter/count/export query builders. Which builder ran depended on whether a namespace happened to be selected when the app probed the driver, and the plugin's copy guessed literal types on an engine with no implicit conversions. The app's TableQueryBuilder and FilterSQLGenerator now build every R2 query. The health monitor is off, following Snowflake's precedent for a stateless HTTPS API. The dead r2Namespace read is gone.
  • Metadata. defaultSchemaName is now "". The plugin used to inherit PluginKit's public default, which overrode the curated value once the plugin loaded.
  • One metadata source. The plugin reads its metadata from CloudflareR2SQLMetadata.swift. The same file is compiled into the test target, and CloudflareR2SQLMetadataParityTests holds the app's curated snapshot to it.

Merge of main

Merged twice.

4. fix(plugin-beancount): BQL is a read, and Beancount is held at Read-Only

Before: Beancount is a read-only engine: the driver refuses anything but SELECT, WITH, EXPLAIN, two PRAGMA forms and BQL:. But QueryClassifier read BQL: BALANCES as SQL and classified it as a write, so holding Beancount at Read-Only would have refused every BQL query. That is why the review had dropped Beancount from the enforcement.

Now: QueryClassifier reads the prefixes the driver accepts (bql:, bql , pragma table_info, pragma database_list) as reads, on Beancount only. Beancount's curated capabilities mark it isEngineReadOnly, so it runs at Read-Only like R2 SQL. A write typed against a ledger is still classified as a write, and Read-Only refuses it before the driver does.

5. fix(connections): the managed Safe Mode minimum shows in the UI

Before: a configuration profile's minimumSafeModeLevel (#2214, shipped in 0.67.0) was applied only inside the execution gate. The toolbar padlock, Database > Safe Mode Level, the connection form and every other reader showed the user's own weaker level. The grid offered inline edits under a Read-Only profile, and the gate refused them only at save. docs/features/safe-mode.mdx already said the control appears dimmed; it did not.

Now: ReadOnlyEnforcement becomes SafeModeFloor, a level plus a reason (readOnlyEngine, remoteDatabaseFile, managedPolicy), because a profile minimum and a read-only engine are the same thing: the weakest level the connection may run at. DatabaseConnection.safeModeLevel raises the user's level to that floor, so every reader gets the profile's minimum with no change of its own. The Safe Mode menus dim every level below the floor. The form lists only the levels at or above it, shows the level in force, and explains why in the footer. A single allowed level still shows as fixed text. Picking the level already in force changes nothing, so the saved level comes back once the profile is removed. The strictness order moves onto SafeModeLevel, and the gate keeps its own read of the profile so a profile installed mid-session applies at once.

The form lists the allowed levels rather than dimming the rest. A probe could not confirm that SwiftUI dims a single item in a macOS menu Picker: no NSPopUpButton surfaced in the hosting view to check. The menus use validateMenuItem, which is proven.

Before / After

Screenshots are pending and need a follow-up. The Safe Mode row and the status bar both need a Debug build driven on screen, and the R2 table tab needs a live R2 account. Neither was available in this session, and another session was running UI tests on this machine. States to capture: the connection form's Options pane for R2 and for a remote SQLite file (fixed Read-Only row with its footer), the Safe Mode menu with the other levels dimmed, the form and menu under a minimumSafeModeLevel profile, and an R2 table tab's status bar with the page buttons gone.

Verification

  • swift test --package-path Packages/TableProCore --filter TableProR2SQLCoreTests: 31 tests in 7 suites pass. They cover the descriptor envelope, strict decoding, exact decimals, base64 bytes, nested JSON, DESCRIBE read by name, both SHOW shapes, error classification, and the transport cancel behaviour against a URLProtocol stub.
  • xcodebuild test on the touched suites, each commit on its own:
    • Commit 1: ReadOnlyEnforcementTests and the safe-mode, sync, storage, form, gate, MCP and import suites (233 + 28 cases).
    • Commit 2: PaginationCapabilityTests, LeadingRowsStatementTests, RowCountPlanTests, the ResultStatusModel, menu validation, TableQueryBuilder, QueryTab and Rewind suites (99 + 174 + 56 cases).
    • Commit 3: CloudflareR2SQLMetadataParityTests, HealthMonitorOptOutParityTests, the registry and classifier suites (188 cases).
    • Sections 4 and 5: SafeModeFloorTests, QueryClassifierBeancountTests, ManagedPolicyResolverTests, ExecutionGateTests, ConnectionFormEditsCoverageTests (79 cases).
    • After the second merge: the registry count, both parity suites, SafeModeFloorTests, QueryClassifierBeancountTests, LeadingRowsStatementTests and PaginationCapabilityTests, plus DatabaseTypeTests in the package.
    • All pass.
  • xcodebuild -scheme CloudflareR2SQLDriverPlugin build: BUILD SUCCEEDED.
  • The AllPlugins aggregate fails locally in the third-party OracleNIO checkout (a @TaskLocal macro expansion under the Xcode beta toolchain). No file this PR touches is involved.
  • swiftlint --strict on every touched path: clean, apart from two findings that were already in SyncRecordMapperConnectionTests.swift. The docs checks pass.
  • Security review: no findings.
  • Code review: Codex was out of credits (usage limit, until Sep 15), so the review ran through the code-review skill instead. Its five findings, and what happened to each:
    1. Holding Beancount at Read-Only broke BQL, because BQL: … classifies as a write. Beancount was dropped from the enforcement then. Section 4 fixes the classifier and holds Beancount at Read-Only again.
    2. Picking Read-Only on a held connection overwrote the saved level. Fixed with chooseSafeModeLevel.
    3. A result that reached the 10,000-row ceiling read as complete. The driver now says where it stopped.
    4. A query-result export at the ceiling had no warning. Added.
    5. Data compare over R2 SQL: not reachable. Compare requires .dataCompare, which this driver does not declare.

Not verified: a live R2 SQL round trip. No R2 account was available. Before the plugin release, please run:

npx wrangler r2 sql query "<account>_<bucket>" "SHOW TABLES IN <namespace>"

That run confirms the name column the table listing reads. SHOW NAMESPACES is the other statement whose output nothing documents.

No UI automation: registry plugins never load in the UI test host, and there is no R2 server to talk to.

https://claude.ai/code/session_01JKFSBk6YwDemnkbQnyc2xz

@mintlify

mintlify Bot commented Aug 6, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
TablePro 🟢 Ready View Preview Sep 11, 2026, 1:40 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Signed-off-by: Ngô Quốc Đạt <datlechin@gmail.com>
Signed-off-by: Ngô Quốc Đạt <datlechin@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant